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Abstract. We combine incentive, adaptive, and time-scale dyan- 
mics to study multipopulation dynamics on the simplex equipped 
^ with Riemmanian metrics, simultaneously generalizing and extend- 

ing many dynamics commonly studied in dynamic game theory 
and evolutionary dynamics. Each population has its own geom- 
etry, method of adaptation (incentive), and time-scale. Using 
information-theoretic measure of distance we give a widely-applicable 

I— I Lyapunov result for the dynamic. We include a wealth of examples 

^0 leading up to and beyond the main results. 

I 

^ 1. Introduction 

Evolutionary dynamics now includes the study of many discrete and 
^ continuous dynamics such as the replicator dynamicj31j, the best re- 

0\ ply dynamic [14j, the projection dynamic |23| |29] |20|, the logit dy- 

namic to name a few. We explore the unification of all these 
i/^ dynamics using the game-theoretically motivated incentive dynamic 

^ introduced in |11|, in discrete and continuous forms using time-scale 

calculus [4j, with Riemannian geometries on the simplex, specified by 
a geometrically motivated parameter called an escort [T5| that allows 
^ some dynamics to be realized in multiple ways and by an arbitrary 

• fh Riemannian metric in general. Moreover, we show that the incentive 

^ dynamic and the replicator dynamic are equivalent, through a map- 

ping that yields insight into the stability of the forementioned dynam- 
ics, and explains clearly how to separate the selection action from the 
underlying geometry, allowing for e.g. best-reply projection dynamics. 
Ultimately we define the time-scale escort incentive dynamic and time- 
scale metric dynamics (which in the continuous case correspond to a 
special case of the adaptive dynamics of flT)), building up through a 
series of examples, and prove a general stability theorem for a large 
class of discrete and continuous dynamics. For general introductions 
to evolutionary dynamics see ^I9j [7j [18j . 
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In this paper we show that the KuUback-Leibler information diver- 
gence and natural generahzations serve as Lyapunov functions in a vari- 
ety of contexts. The use of the KL-divergence and similar information- 
theoretic quantities in evolutionary dynamics goes back at least to ^ 
and is developed further in |32|. A geometrically-motivated general- 
ization from information geometry |1| was introduced in fTE\. To the 



reader familiar with information geometry, this should come as little 
surprise since the Shahshahani metric of evolutionary game theory can 
be indent ified with the Fisher information metric, which is in some 
sense a local variant of the KL-divergence. The projection dynamic 
can similarly be described in terms of the Euclidean geometry [21] and 



stability in terms of the Euclidean distance 24 which can be realized 
as a generalized information geometry [15] through the introduction 
of a functional parameter called an escort. We show in this work that 
the geometry can simplify the stability theory of some evolutionary dy- 
namics in addition to defining new dynamics, and is compatible with 
the formulation of the incentive dynamic and the time scale calculus. 
Finally, we introduce an information divergence for a general Riemann- 
ian metric satisfying some mild assumptions, extending the adaptive 
dynamics of [Tt] to discrete time scales. Together, these ingredients 
yield a very general stability result. 

2. Incentive Dynamics 

Let us first introduce the incentive dynamics of Fryer [TT]. Moti- 
vated by game-theoretic considerations, the incentive dynamics takes 
the form 

(1) Xi = (fi{x) - Xi^(fj{x). 

3 

Table [T] lists incentive functions for many common dynamics. Fryer 
shows in ^T] that any game dynamic can be written as an incentive 
dynamic and gives a stability theorem as follows. Define an incentive 
stable state (ISS) to be a state x such that in a neighborhood of x the 
following inequality holds 

^2>| ^ XiLpi{x) ^ ^ Xi(fi{Xi) 

Xi . Xi 

It is then shown that given an internal ISS, the KuUback-Liebler infor- 
mation divergence is a local Lyapunov function for the incentive dy- 
namic (Equation [T} The incentive (pi{x) = Xifi{x) captures the known 
result for the replicator dynamics [5] [Ts], with the definition of ISS 
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being exactly ESS: x • /(x) > x • f{x). Moreover, a short proof shows 



that for the best reply incentive, ISS is again ESS 12 



Dynamics Name 


Incentive 


Replicator 


(fi{x) = Xi {fi{x) - f{x)) 


Best Reply 


ipi{x) = BRi{x) - Xi 


Logit 


. ^ _ exp(77-V,(x)) 


Projection 


^i{x) = < 


j^S{f{x),x) 

else 



Figure 1. Incentives for some common dynamics, 
where BRi[x) is the best reply to state x, S{f{x)^x) 
is the set of all strategies in the support of x as well 
as any collection of pure strategies in the complement 
of the support that maximize the average. Note that 
on the interior of the simplex the projection incentive is 
just (^i{x) = fi{x) — ^/"^^j fj{x). For more examples 
see Table 1 in [llj . 



2.1. The Incentive Dynamic is the Rephcator Dynamic. Ob- 
serve that we can transform any incentive dynamic into a replicator 
dynamic on the interior of the simplex (the behavior of incentives 
near the boundary of the simplex is another matter altogether and 
we will not consider it here). We simply solve for gi in the equation 
^i{x) = Xifi{x) so that to every incentive (fi we define an effective 
fitness landscape = which is well-defined at least on in- 

terior trajectories and possibly on the boundary simplices, such as is 
the case for forward-invariant dynamics. (Note that some incentives 
take particular care on the boundary, such as the projection incentive 
in |29|.) The summation term (fj{x) in equation ([T]) is the mean of 
the effective fitness landscape. 

The ISS condition for an incentive is the same as the ESS condition 
for the effective fitness landscape, and this shows that the ISS stabil- 
ity theorem is equivalent to the analogous theorem for ESS and the 
replicator dynamic. This does not of course imply ESS for any fitness 
landscape used in the definition of an incentive (such as a best reply 
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incentive using a landscape /). Typical fitness landscapes in evolution- 
ary dynamics are linear and given by f{x) = Ax where A is a game 
matrix. In this formulation, one will encounter a much larger class 
of landscapes. For the best reply dynamic, the effective landscape is 
fi{x) = BRi{x)/xi — 1, and while it is clear that such a function may 
not be well-defined on the boundary simplex, since the dynamic is 
forward-invariant we will not concern ourselves at this time. 



2.2. Example: q- Replicator Incentive. From the preceeding sec- 
tion it is tempting to suspect that the concepts of ISS and ESS are 
equivalent, but this is not the case. Consider the q-replicator incentive 
(fi{x) = x^fi{x) for a fitness landscape /. Further, assume that the 
fitness function is of the form /(x) = Ax where A is the rock-scissors- 
paper matrix: 











X 



y—b a y 

Several curves for various values of q are plotted in Figure [2j For the 
replicator incentive {q = 1) the trajectory converges to an interior ESS; 
for other values of g, the trajectories may either converge or diverge. 
This shows that an ESS for the fitness landscape need not be an ISS for 
the incentive. Note also that whether a curve converges or not depends 
on the initial point. While for g = 1 the Lyapunov function is global, 
this is not the case for other values of q. See Figure |3| Figure |4] shows 
that an ISS need not be an ESS, and Figure |6] shows that an ESS need 
not be an ISS. 



3. Time Scale Incentive Dynamics 

3.1. Time-Scale Definitions. To study dynamics at diflFerent time 
scales, a key ingredient is the time scale derivative, also known as the 
delta derivative. General references for time scale dynamics include [3] , 
|4]. For time scales T = /iZ, the time scale derivative is given by: 

.A _ /(x + h)- f{x) 
^ h 

which is better known as the diflFerence quotient. For the time scale 
T = M, the time scale derivative is the standard derivative. While there 
are other common time scales, this paper restricts attention to M and 
/iZ for < /i < 1 for the most part. 
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(a) Initial point: (1/10,1/10,8/10) (b) Initial point: (1/83,2/83,80/83) 

Figure 2. Phase portraits for q- Replicator incentive for 
the powers q G {0.5,1,1.5,2,2.5,4} with colors blue, 
green, red, cyan, magenta, and yellow, respectively. The 
game matrix is the the RSP matrix with a=-l, b=-2. An 
ESS need not be an ISS since some of these trajectories 
converge and some do not, depending on the incentive. 



3.2. Time Scale Replicator Equation. Before we proceed, let us 
consider an illustrative example. Define the time scale replicator equa- 
tion as: 



Note that J2i = 0. For /i = 1, this is the discrete replicator dynamic, 
where 

^ x{t + h) — x{t) 



rp — • rp rp . 



which can be easily seen to be equivalent to the usual discrete dynamic 

^ifi (^) 



X- 



For the case /i — )■ 0, we have the following: 

Xi {fi{x) - f{x)) 



Xi 



fix) 

This equation is trajectory equivalent to the replicator equation (up to 
a change in velocity and transformation of /(x) so that /(x) > jisj 
[7]). The definitions of the time-scale calculus can unify the description 
of continuous and discrete dynamics in evolutionary dynamics, as well 
as setting the stage for dynamics on other time scales, through the 
delta derivative. 
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■ 0.00000000 
1-0.01357004 

}.02642587 
1-0.04050607 

-0.05356596 
-0.06652382 
-0.07999184 
-0.09927558 
- -0.12774207 
-0.17477808 



(a) q = 0.78 



(B) g=l 




0.02908028 



-0.03449792 



(c) g = 2.5 



Figure 3. ISS: LHS - RHS. Stability need not be global 
for an ISS. Note that for g = 2.5, the quantity changes 
sign depending on point. This is true for all g > 1. 



3.3. Time Scale Best Reply and Fictitious Play Dynamic. In 
(n), a best reply dynamic is shown to result from the incentive Lpi{x) 
BRi{x) — and the ISS condition is shown to reduce to the ESS 
condition. Now consider the case of T = KL for h G (0, 1). This time 
scale yields a discrete best reply that is algebraically equivalent to 

X- = (1 - h)xi + hBRi{x). 

Here we see that the time-scale appears as a weighting between the best 
reply and the current state of the dynamic. In other words, h is the 
proportion of the population that adopts the best reply. If /i = 1, then 
entire population switches to the best reply, and the dynamic cycles 
through the corner points of the simplex for the landscape defined by 
an RSP matrix as above. 

The center of the simplex is not an equilibrium point for the time- 
scale T = KL. To see this, suppose the fitness landscape / has an 
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Figure 4. Phase portraits for q- Replicator incentive for 
the powers q G {0.5,1,1.5,2,2.5,4} with colors blue, 
green, red, cyan, magenta, and yellow, respectively. The 
game matrix is the the RSP matrix with a=l, b=2. Ini- 
tial point: (1/8,1/8,6/8). The standard replicator dy- 
namic diverges for this landscape, so an ISS need not be 
an ESS. 

interior ESS at the barycenter (say for an RSP matrix). On the interior 
of the simplex, the dynamic is stationary if and only if x/^+i = Xk for 
some /c, which implies that either /i = or G {0, 1} for alH, which is 
impossible for an interior ESS. However, if the time-step h is not fixed, 
this may not be the case (see [7| . On a variable time scale the dynamic 
is (in vector form) 

Xk+i = (1 - hk)Xk + hkBR{xk), 

which is similar to the fictitious play dynamic in the case that the 
time-scale weights decrease over time. Eventually such a time scale 
has /ijt ^ 0, so the above discussion does not apply, and this dynamic 
can converge. 

3.4. Time Scale Lyapunov Function. Lyapunov stability is a frequently- 
used technique in evolutionary dynamics. These techniques have been 
ported to the time scale calculus ^ [sj [9j . To the reader familiar with 
the traditional theory, the methods will be very familiar. A time scale 
Lyapunov function is a positive definite function defined on the trajec- 
tories of a dynamic with negative-semi-definite (negative-definite) time 
scale derivative which then implies stability (asymptotic stability). The 
interested reader should see ||2j for a presentation that mirrors the tra- 
ditional approach with the appropriate changes necessary to extend the 
stability theory to arbitrary time scales. In particular, we will use The- 
orems 5.1 and 5.4 in l2], which have two technical conditions that will 
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Figure 5. Best reply for /i = 1/3 (blue) and a variable 
time scale with hk = + 1) (green). Only the latter 
converges to the interior ESS. 

be familiar to the reader aware of the details of the continuous-time 
Lyapunov stability theorems. Here we give the necessary time-scale 
definitions and discuss the technical conditions; for more detail, see 
Define Kh{x) = {x : \\x-x\ \ < h}. A function c : [0, h] [0, oo) 

belongs to class K if it is continuous and increasing, and if c(0) = 0. 
A function v : x T ^ M is called decrescent if there exists a func- 
tion c of class K and to G T such that for all t > to and x G 
v{x^t) < c{\x — x\). Under the same conditions, if there exists c such 
that v{x^t) > c{\x — x|), V is called positive definite. We will take the 
definition of time scale Lyapunov function to include both conditions 
on the appropriate time scale; all such Lyapunov functions in this paper 
satisfy both conditions. 

We now define a time scale generalization of the incentive dynamics 
and generalize the Lyapunov results of \TT\ and [T5| to cover more 
general time scale dynamics. 

Theorem 1. For < h < 1^ D{x) = D{x^x) is a Hime-scale Lya- 
punov function^^ for the time scale incentive dynamic iff x is an ISS. 

This theorem is a special case of Theorem [3] and so the proof is de- 
ferred. A direct proof for the replicator equation is an straightforward 
exercise. 

4. ISS AND ESS 

A natural question is: for which incentives, (f{x)^ are the stability 
concepts ISS and ESS equivalent? Since an interior ISS is unique and 
asymptotically stable for the incentive dynamics and an ESS is unique 
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and asymptotically stable for many known special cases (e.g. the repli- 
cator dynamic) , either the concepts must coincide in these cases or only 
one of the two criterion can hold. 

Without loss of generality, we assume (Pi{x) = for every x G A. 
The ESS condition x • f{x) < x • f{x) for all x in a neighborhood of 
X can be rewritten as x • [f{x) — {x • /(x))l] > 0, where 1 is a vector 
of all ones. Thus at a pure ESS, the payoff positivity requirement, 
ipi{x)/xi > <^ fi{x) > X • /(x), preserves not only the asymptotic 
stability but also the exact basin of attraction as well. Furthermore, 
the set U = {fi{x) > x • f{x)}f^j_^-{fj{x) < x • f{x)} is open and 
non-empty for a continuous fitness near a pure ESS. Thus weak payoff 
positivity [32], where at least one above average fitness must have a 
positive incentive growth rate, preserves the asymptotic stability of a 
pure ESS, albeit for a smaller basin of attraction. 

Samuelson and Zhang |27| introduced a class of games called aggre- 
gate monotone^ which trivially preserve ESS and the basin of attrac- 
tion. However, they prove in the single population case all aggregate 
monotone dynamics are dynamically equivalent to the replicator equa- 
tion. In the multi-population case the dynamics for each population 
is a positive function multiplied by the replicator equation. In either 
case it does not lead to interesting further discussion. 

Definition 1 (Aggregate Monotone). An incentive is aggregate mono- 
tone ifyy^ z G 

y • /(x) > z • f{x) ^ y • ^ > z • ^ 

Attention may be restricted to single population games with mixed 
ESS, as Selten [SO] established that an ESS in an asymmetric gam^ 
must be pure. This result assumes linear fitness which is more restric- 
tive than necessary for the arguments above. However, the following 
results do all assume linear fitness. 

First introduced by Nachbar |22|, the concept of payoflF monotone 
dynamics is a reasonable starting point for investigation. 

Definition 2 (Payoff Monotone). An incentive is payoff monotone if 

/.w > /,M « ^ > ^eiM 

Xl Xl J 

Unfortunately, Friedman |10|, via a verbal description, shows that 
payoff monotone dynamics need not be asymptotically stable at an 
internal ESS. 



The asymmetric condition includes all multi-population formulations. 



10 



MARC HARPER AND DASHIELL FRYER 



Fortunately, Cressman |6| shows that for smooth incentives the con- 
dition of payoff positivity wiU preserve the asymptotic stabihty of an 
ESS as long as the linearization is not trivial. His results are para- 
phrased in this theorem for completeness. 

Theorem 2. An interior ESS, x, is asymptotically stable for any 
smooth payoff positive dynamic which has a nontrivial linearization 
about X. 

5. Escorts 

The stability theorem for the incentive dynamic reduces the prob- 
lem of finding a Lyapunov function to the problem of proving the ISS 
condition valid for a candidate equilibrium. Consider the projection dy- 
namic, with incentive (^i{x) = fi{x) — \ fj{x). From |21| and |29j, 
we know that | |x — x| p is a Lyapunov function if x is ESS, so one would 
hope that ISS reduces to ESS in this case as well. The ISS condition 
{x/x) • f{x) > 1 • /(x), however, is not obviously the same as the ESS 
condition. If we assume a two-type linear fitness landscape 

m = 

it is a straightforward derivation to show that the ISS condition leads 
to 

[(a — c)xi + {b — d)x2] {x2Xi — X1X2) > 0, 
which along with the constraint that Xi + X2 = 1 is the well-known 
condition for the existence of an internal ESS (and more generally, 
a Nash equilibrium). Similarly, we can give a family of examples that 
includes the projection dynamic. Consider the escort dynamics defined 
in where is a positive non-decreasing function: 




(3) Xi = (l)i{x) fi{x) - 



For these dynamics, an incentive is given by the right-hand-side of 
Equation |3] and the ISS condition is 



E Xi(pi{x)fi{x) ^ Xi(j)i{x)fi{x) 



which only clearly reduces to ESS for the replicator dynamic ((/>(x) = 
x). Nevertheless, it was shown in fT5] that an ESS, if it exists, is 
asymptotically stable for these dynamics, so it is desirable to have a 
generalization of Theorem [2] that captures this family as well. 
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To this end we introduce a functional parameter called an es- 
cort and related quantities. An escort is a function (j) that is non- 
decreasing and strictly positive on (0,1]. A vector- valued function 
(/){x) = (01 (x), . . . , (/)^(x)) is called an escort if each component is an es- 
cort. We denote the normalized escort vector by 0, i.e. normalized such 
that (/)(x) is in the simplex. Generalized information divergences are 
defined by generalizing the natural logarithm using an escort function. 
See and [25] for more details. 

Definition 3 (Escort Logarithm). Define the escort logarithm 



The escort logarithm shares several properties with the natural log- 
arithm: it is negative and increasing on (0, 1] and concave on (0, 1] if 
(j) is strictly increasing. 

Definition 4 (Escort Divergence). Define the escort divergence 



Since the logarithms are increasing on (0, 1), this divergence satis- 
fies the usual properties of an information divergence on the simplex: 
D{x^ y) > if X y and D{x^ x) = 0. 

6. The Time Scale Escort Incentive Dynamic 

Now we are able to define the time scale escort incentive dynamic 
and give the main theorems. 

Definition 5. Define the time scale escort incentive dynamic to be 



We also need a definition of ISS that incorporates the escort param- 
eter. 

Definition 6. Define x to be an escort ISS (or EISS) for an incentive 
and an escort (j) if for all x in some neighborhood of x the following 
inequality holds: 





1=1 




3 
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Remark: This definition can be understood in terms inner products 
of a Riemannian metric gij{x) = 5ij/(f)i{x). 

For one example of a choice of escorts leading to interesting dynam- 
ics, consider the escort (/)(x) = (3x. This introduces an intensity of 
selection parameter that alters the velocities of the dynamic (but not 
the stable point, if any). In fact, each type can have its own intensity 
of selection, and its own geometry. A popular choice of escort in in- 
formation geometry is (j){x) = x^, which gives q-analogs of logarithms, 
exponentials, and divergences ^25j. See y_5| for more examples. The 
corresponding q-divergence is given by: 



D,{x\\y) 



\x 



211- y\\ 

DKLix\\y) 

Xi 



1 

l-q 



log 



+ 



■ 2 — q 2 — q 
v ■ —X- 



y] \yi-Xi) 



2-q 



if g = 

if g = 1 

if g = 2 

if g> 0,9 7^ 1,2 




Figure 6. q-Escorts for various g, with replicator incen- 
tive given by a linear fitness landscape with RSP game 
matrix (a=l, b=2). Left: q G {0.2, 0.8, 1.0, 2.0} with col- 
ors blue, green, red, and cyan respectively; Right: g = 4. 
The standard replicator equation diverges for g = 1 but 
this is not always the case for the q-escort. 

As in the case of the incentive dynamic, the continuous-time escort 
incentive dynamic is a special case of the escort replicator dynamic. 
Consider the escort exponential exp^, which is the functional inverse 
of the escort logarithm. It has the following crucial property: 

-^exp^x = (/)(exp^x) 
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Now let Xi = exp^.{vi—G) and consider the derivative Xi = (/){exp^.{vi- 
G)){vi — G) = (l)i{x){vi — G). Then if we can solve the following two 
auxiliary equations we can also solve the escort incentive dynamic: 



G 



(t)i{x) 



This also shows how we can translate the escort incentive into the 
escort replicator equation. Given the escort incentive 

Xi = ipi {x) - (x) ^ (fj (x) , 

j 

define a fitness landscape 

•^•<-' = fM- 

Then the escort incentive equation is an escort replicator equation, and 
the ESS condition x • /(x) > x • /(x) is the EISS condition 

V ^^(^) ^^(^) ' 

That an x is ESS iff the escort divergence is a Lyapunov function 
was shown in |15j for the continuous escort replicator dynamic. What 
remains to be shown now is the extension of this result to time scales 
T = /iZ for < /i < 1, and the relationship between the concepts of 
escort ISS and ESS. Before doing so, consider the following examples. 
In Figure |6| replicator incentives for RSP matrices are plotted with 
various g-incentives. Notice that the equilibrium is not an ESS for 
the fitness landscape / yet the dynamic converges to the center of the 
simplex for some choices of the escort. In Figure [7| we have phase 
plots for dynamics with the same landscape but with both q-replicator 
incentives and q-escorts. In this case the conditions for E-ISS and ESS 
are the same, so all the dynamics converge. The KL-divergences are 
not monotonically decreasing in all cases, but the corresponding escort- 
divergences are in fact Lyapunov functions, as shown in Figure |8| The 
dynamic in this case is given by: 

(4) i;, = x?/,(x)-^^a;j/,(a;). 

Now we state and prove the main theorem of this section. 
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Figure 7. Plots of Equation [i] for various q. Left: 
0.5,1.0,1.5,2.0 blue, green, red, cyan, magenta, repsec- 
tively; Right: g = 4. 




Figure 8. Plots of candidate Lyapunov functions for 
q-escort and q-incentive. Left: KL-divergence for all dy- 
namics; Right: q-divergences for the corresponding q for 
each dynamic. Colors correspond to the dynamics in 
Figure [7] 

Theorem 3. Let Lp he an incentive function and let (j) he an escort. If 
X is an escort ISS then D{x) = D^{x^ x) is a local time scale Lyapunov 
function for the time scale escort incentive dynamics. 

The proof follows easily from the established facts and the following 
lemma. 

Lemma 1. 



D^{x)<-Y, 
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Equality holds in the limit that h ^ 0, i.e. for T = M. 

Proof. Since escort functions are nondecreasing and positive on (0, 1], 
we have the foUowing two facts: 



/ 

J a 



^ du b — a 
< 



[u) (f){a) 
log^{u)du < (6-a)log^(6) 

J a 

Now we have 

^ rx'^ ^ 

hD^{x) = XI / ^^S</>(^) du-Y^ [{x\ log^^ x\ - Xi log^^ x\) - {xi log^^ Xi - Xi log^^ Xi)] 

^"^(^i- log</>(^i) - ^^S^^ ~ ^^S</>, ^'i - Xi log^^ Xi + Xi log^^ Xi 

i i 

= (£i - Xi) (log^^ X'i - log^^ Xi) 

i 

du 



[ Xi Xi 



Xi (t>i{u) 



= -E( 

i 



Xj Xj 



Bringing h to the right hand side completes the lemma for /i > 0. 
Equality for /i = (i.e. T = M) can be directly verified with differenti- 
ation (and is given in |15|). □ 

To complete the proof of Theorem [3] we need only apply the lemma 
to the respective dynamics, use the definition of EISS, and verify the 
two technical conditions of the time scale Lyapunov theorem. 

Proof of theorem \^ Using the lemma and substituting the right-hand 
side of equation [5] gives: 

D''{x)<-Y,^'~''' 



A 

Xi 



(i)i{x) 



X) 



= (Xi - Xi) {fi{x) - f{x)) = X ■ f{x) - X ■ f{x) 

where / is the effective landscape. If x is an EISS, then the right-hand 
side is negative. □ 
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Throughout this paper, it may appear that aU the examples we have 
given are for continuous dynamics. In fact, every phase plot in this 
paper is for a discrete time-scale with h ^ 1/100 1/1000 unless 
otherwise indicated, and in all previously known cases, the results have 
coincided with the expectation for the continuous dynamics. Hence 
all the examples in this paper illustrate the main theorem for these 
particular time-scales. 

Let us return to the example of the projection dynamic. Previously 
we saw that the orthogonal projection dynamic was obtainable from the 
incentive (fi{x) = /^(x) — (1/n) fj{x) and the escort (/){x) = x on in- 
terior trajectories, but the ISS condition did not appear to be the same 
as ESS (though we were able to argue for two-player linear landscapes 
that they are equivalent). With the generalized theorem, we can also 
obtain the dynamic from the incentive (fi{x) = fi{x) and = 1, 

and now the EISS condition reduces immediately to ESS. Moreover, 
the Lyapunov function given by the escort information divergence is, 
remarkably, one-half the Euclidean distance: D{x) = (l/2)||x — 
capturing the known result up to a constant factor of one-half f24] [21^. 

From the these results it should be clear that if the incentive factors 
as (fi{x) = (j)i{x)gi{x) then the condition for EISS will be the con- 
dition for ESS for the landscape g. So in particular if the function 
g is payoff monotonic, these dynamics have similar interior dynamics 
(though with different Lyapunov functions). If the incentive factorizes 
as (^i{x) = Xi(l)i{xi)gi{x)^ then the ESS criterion is the ISS criterion 
for the incentive g. So while the theorem covers all combinations of 
valid incentives and escorts, it is clear that careful choices may lead 
to simplifications in the stability criterion. The projection dynamic 
is somewhat special in that it can be described equivalently by mul- 
tiple choices of escort and incentive. It is also clear that while there 
exist known Lyapunov functions for some dynamics that are not the 
KL-divergence (or a generalization) such as the best reply dynamic, the 
incentive dynamics approach yields a commonly-derived and motivated 
Lyapunov function. 

6.1. A Variation of the Replicator equation. We can also define 
novel dynamics by combing escorts and incentives. Consider the fol- 
lowing dynamic, using the replicator incentive (^i{x) = Xifi{x) and the 
escort (f)i{x) = 1: Equation ^ gives the dynamic as 

Xi Xifi(x^ f (x\ 

n 

In this case the EISS criterion is ^^XiXifi{x) > ^^XiXifi{x)^ which is 
the same for the dynamic obtained with Lpi{x) = x'^fi{x) and (j)i{xi) = 
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Xi. These two dynamics differ qualitatively, however, as we can see for 
a constant fitness landscape fi{x) = 1 for all i. While both dynamics 
have the barycenter of the simplex of rest points, the former is not 
forward-invariant on the interior and has no other rest points, while 
the latter has rest points on the boundary and the barycenters of the 
boundary simplices. Moreover for an RSP matrix, while the replicator 
equation converges to an interior ESS, the projection variant does not. 
In general, this dynamic may not be forward invariant on the simplex 
since while the sum J^i = 0, the individual components can become 
negative for certain states and fitness landscapes. 

6.2. Best Reply Variants. Similarly, we can define a projection best- 
reply dynamic using p-escorts. See Figure [9| Note that for p = 0.5, 
the dynamic is not forward-invariant on the interior. Starting near 
the barycenter yields "shapley-like-triangles" . These can be extended 
cycles that jump on and off the boundary if the best reply incentive is 
combined with the projection incentive (cyan curves in Figure|9]). Using 
just the projection escort (i.e. q = 0) would not yield this behavior 
(trajectories remain on the boundary once they reach it). 



Figure 9. Left: Best reply incentives with g = 0.6, 1, 3 
and combined with projection incentive at initial point 
(1/10,1/10,8/10) blue, green, red, respectively; Right: 
Same with initial point (1/3.1, 1/3.1, 1.1/3.1). 

6.3. Incentive-Escort Dynamic is just the Incentive Dynamic; 
Zero-Sum Incentives. We have seen that the incentive-escort dy- 
namic can be rewritten as the escort dynamic; it can also be rewritten 
as the incentive dynamic by simply taking the right-hand side as the 
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incentive. As we have seen, however, one may miss the stabihty of an in- 
ternal equihbrium by trying to apply the KL-divergence as a Lyapunov 
function since it is not necessarily monotonic. So while the decompo- 
sition of an incentive dynamic into an incentive-escort dynamic does 
not yield a larger class of dynamics, it does yield stability theorems for 
more members of the class. 

Zero-sum incentives, when ^i{x) = are informative here. First, 
notice that in this case the escort-incentive dynamic reduces to just 
the incentive dynamic, producing the exact same trajectory for any 
escort. One such example is the replicator incentive for a RSP matrix 
with a = 1 = b. On the other hand, any incentive can be made into a 
zero sum incentive by subtracting y {"^i^iix)) from the incentive for 
any value of y in the simplex, and the choice of y can vary at each 
point in the simplex. In other words, it requires a choice of escort, or 
more generally, a choice of Riemannian metric. In this case, there is a 
preferred choice of incentive, and so a natural candidate for a Lyapunov 
function. 

7. Time-Scale Lyapunov functions for Adaptive Dynamics 

As suggested in the last section, we now consider the adaptive dy- 
namics for a Riemannian metric G on the simplex, defined in ^I7j. Let 
g = G~^l and C = G~^ — {g^)~^gg^- Then the time-scale adaptive 
dynamics for the metric G and a fitness landscape / is = Cijfj. 
The adaptive dynamics includes the escort dynamics as a special case. 
A geodesic approximation is shown to be a local Lyapunov function 
in |17| for the continuous dynamic; to obtain a time-scale Lyapunov 
function we use a global divergence, which we now give define. Let 



Theorem 4. Let G be a Riemannian metric and assume that the re- 
ciprocal of each component Gij is an escort function. Then 

(1) Dg is an information divergence; that is Dg{x) = and Dg{x) > 
for X X 

(2) D is a local time scale Lyapunov function for the adaptive dy- 
namics if X is an ESS. 




where 
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The proof is almost identical to the proof of theorem [3j and so is 
omitted. Note that Dg{x) < (x — — x). One can now 

play the same game as before, identifying the incentive in terms of the 
fitness landscape, and forming best reply, logit, projection, or any other 
incentive dynamics with respect to particular Riemannian geometries. 
Indeed, we have that (^i{x) = G^j^fj{x) and the adaptive dynamics 
can be written as 

(5) = - J^^^Q^}[^ . ^^■(^)- 

Call this the metric-incentive dynamic. As before, we can identify 
the idea of an ESS with that of a G-ISS, i.e. a state x is a G-ISS if 
for all X in a neighborhood of x, the inequality Lp{x) • G{x){x — x) > 0. 
Then we these definitions, we can restate Theorem [i] (2) as follows. 

Theorem 5. Let G be a Riemannian metric and assume that the re- 
ciprocal of each component Gij is an escort function. Then 
Dg is a local time scale Lyapunov function for the metric-incentive 
dynamics if x is a G-ISS. 



8. Multiple Populations 

Following equation [5| we can formulate a multiple population dy- 
namic in which each population operates on its own incentive, time- 
scale, and geometry. Let G{x) = y!^^(g-^\) be the vector of coefficients 
in equation [5) Then the multiple population time-scale metric incentive 
dynamic is (populations indexed by a)\ 

(6) X"^" = (fi^a{Xa) - Gi^a{Xa) ^ ^jA^) 

j 

8.1. Examples. We give two examples before discussing stability in 



Figures [10] and [TT] The only difference between the two examples is 
the incentive for the second population: in the first case, the incentive 
is logit, and in the second, q-replicator with q = 2. Nevertheless, the 
resulting dynamics are very different. 

In the spirit of |11| and [g], if there is a multiple-population G-ISS 
where the time-scales do not differ for each population then we can 
find a time-scale Lyapunov function for the system by summing Dg for 
each population. The proof is again analogous to |3| and can easily be 
verified with differentiation for the continuous case. 
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Figure 10. Two population dynamic with h = 1/10 
for both populations, with fitness landscape given by 
a = —1, b = —2. Both populations converge to the 
center (500 iterations shown). Left: Replicator incen- 
tive, Shahshahani geometry, initial point (1/5, 1/5,3/5); 
Right: Logit incentive, r] = 0.4, Euclidean geometry, ini- 
tial point (3/5,1/5,1/5) 



Theorem 6. Suppose each population in system^ is of the same time 
scale (T = hZ or T = W). Let L = Dc^ixaWxa)- L is a local 
time-scale Lyapunov function for the system iff 



for some neighborhood of Xa- 

8.2. Variable Time-Scales. A suitable generalization of the time- 
scale Lyapunov theorem that would apply to a system on multiple 
time-scales does not seem to exist in the literature. Rather than prove 
such a result in this paper, we will just give an example relevant to the 
time-scales under discussion, and conjecture further results. 

First consider two populations, on time scales Ti = hZ and T2 = 
h/2Z respectively. We must decide how to compute the derivative of 
the (candidate) Lyapunov function. In this simple case, simply taking 
the intersection of the time scales (resulting in just the the time scale 
Ti and computing the derivative at these points produces a positive 
definite and decreasing quantity, albeit on a subsequence of the origi- 



nal dynamical system. See Figure 12 for an example. The sum of the 
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Figure 11. Two population dynamic with h = 1/10 
for both populations, with fitness landscape given by 
a = —1, b = —2. Neither population converges after 
10,000 iterations. Left: Replicator incentive, Shahsha- 
hani geometry, initial point (1/5,1/5,3/5); Right: q- 
Replicator incentive with g = 2, Euclidean geometry, 
initial point (3/5, 1/5, 1/5) 

appropriate divergences is monotonically decreasing to zero as the pop- 
ulations converge. In this example, if /i = 1/10 for both populations, 
the Lyapunov quantity is not monotonic (it has one local maximum) 
globally, just locally. 

In more general cases, depending on the time-scales involved, it may 
not be so obvious to determine what the minimal or intersection time- 
scale should be (since it could be the case that the intersection is empty, 
especially if there is an interaction between time scales of the form 
T = hZ and say quantum time-scales of the form T = q^). Similarly, 
it may be the case that some type of dot product of time-scale deriva- 
tives with the respective Lyapunov functions for each population is the 
appropriate operation, but this is beyond the scope of this paper, and 
will undoubtedly be discussed in future work. 



The replicator-mutator equation is an evolutionary dynamic that 



9. Mutation 



takes the following form ^6] [26] : 
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(c) 

Figure 12. Two population dynamic with /i = 1/10 for 
first population, /i = 1/20 for the second. Left (a): a = 
— 1, b = —2, replicator incentive, Shahshahani geometry, 
initial point (1/5, 1/5,3/5); Right (b): a = -1, 6 = -4, 
replicator incentive, q = 2 escort geometry, initial point 
(3/5,1/5,1/5); Bottom (c): KL-divergence from center 
of first population plus q = 2 divergence from the center 
of second population. 

where fiji is the mutation transition probability of type j mutating 
into type i at reproduction. Using the relationship (fi{x) = Xifi^ we 
can translate this equation to incorporate incentives: 

3 3 

In vector form, we have that 

X = ip{xY jji — X X^ = ip{xY jji — x{ip{xYl) 

3 
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Finally, we can incorporate time scales and geometric structure into 
a dynamic we call the time-scale metric incentive-mutator dynamic. 

= ^{x)^fi-G{x){^{x)^l) 

A common form of the discrete replicator mutator equation 26 can 
be obtained using h = 1 and the incentive 

This gives: 

A common choice for mutation matrix is /x^ = (1 — e)/^ — 6/(n — 
1)(1^ — /^), i.e. from the identity matrix, subtract e from the diagonal 
and split uniformally over the other n — 1 elements of the row. We 
end with a final example of a best reply dynamic with mutation in 
Figure 13, See [28] for another approach to a best reply dynamic with 



mutation. 



Figure 13. Best reply dynamic with uniform mutation 
matrix, starting at (1/10,1/10,8/10) with fitness given 
by the RSP matrix with a = 1 and 6 = 2. Blue: e = 0.1, 
Green: e = 0.4, Red: e = 0.8. The addition of mutation 
changes the size of the Shapley triangle depending on the 
value of e. 
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10. Discussion 

In this paper we have shown that a vast array of evolutionary dy- 
namics can be analyzed in terms of incentives, Riemannian metrics, and 
divergence functions, the last of which allows the stability analysis of 
many discrete dynamics associated to the same geometric and game- 
theoretic constructions. In particular, we have shown that choice of 
incentive and Riemannian metric can affect the presence of equilibria, 
and that decomposition of dynamics into incentives, escorts, and Rie- 
mannian metrics can lead directly to Lyapunov functions for particular 
dynamics. In the process, we introduced a new information divergence 
defined in terms of a Riemannian metric rather than the typical dy- 
namics formed only from metrics with diagonal matrix representations. 

In our approach to evolutionary dynamics we focused attention on 
a few special cases that yield a particularly nice set of examples. It is 
possible to define these dynamics on arbitrary time scales and achieve 
some analogous results. It may be possible to formulate multipopula- 
tion dynamics where each population evolves according to a different 
time scale, which would require a substantial expansion of the current 
state of stability theorems on time scales. We have also shown that the 
time scale calculus and its stability theory can be a useful and unifying 
addition to the study of evolutionary dynamics. 
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